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TEC: A Manhattan Project in Educational Technoloqv. 
Part II 



WESLEY K. ROBERTS 
US Army Combat Arms Training Board 



In any field of study, significant achievements are 
constantly emerging, yet their impact is often sporadic. 
More than one technology has considerably -improved human 
effectiveness as the result of a singular substantial 
effort. However, few communities, organizations, or instit- 
utions have the wherewithal to initiate large scale projects 
having immediate and long term impact on a technology; apply 
state-of-the-art techniques; aggregate the efforts of a 
variety of interests, agencies, and organizations; or have 
access to a wide representation of project output recipients 
to validate the usefulness of the product before it is ' 
implemented for use by others. . . 

The US Army has historically been a catalyst for rapid 
advancement of a technology for the purpose of meeting both 
immediate and long range objectives. One of these catalyst 
programs was known as the "Manhattan Project". Anyone 
familiar with the outcomes of that effort knows of the^ 
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devastating results that atomic warheads had, both ipimedi- 
ately and permanently. Nonetheless, the Manhattan Project 
achieved considerable advancement for nuclear technology, 
much to the despondency of its opponents and to the gratifi- 
cation of its proponents. 

In August 19 71, the Army began another project, not so 
bold as the one begun during World War II, yet one that has 
served as a catalyst to rapidly advance the field of educa- 
tional technology, even to the degree of being coined as a 
"Manhattan Project in Educational Technology" (Roberts, 
Lawson, and Neal , 1976), This project is called the Training 
Extension Course, or more popularly, by its acronym, TEC. 

The Evolution of TEC 

The TEC project began as a result of a study directed by 
the US Army Chief of Staff, aimed at identifying problems 
leading to ineffective soldiering, and to offer some possi- 
ble courses of action for solution. The group tasked to 
complete this study became known as the Board for Dynamic 
Training. It was formed at Fort Benning, Georgia in 1S71 
and was terminated during the early months of 1972 as its 
functional successor, the US Army Combat Arms Training 
Board, was constituted. 
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The Board for Dynamic Training began its mission using 
both a system and systems analysis (cf. Kaufman, 1972). a 
system analysis identified Army-wide problems. Once isolated 
into generic categories, a systems approach was employed to 
isolate needs. 

Identified needs included a requirement to improve 
training Army-wide (including Reserve Component units) from 
the standpoint of both technical and tactical proficiency 
(cit. Report of the Board for Dynamic Training, 1971). 

Applying both logic and a great deal of cost-effective- 
ness insight to the identified gap between what training was 
being provided and what was desired resulted in a course of 
action to provide prepackaged, self -instructional training 
to soldiers, wherever they happen to be, but primarily to 
those in tactical units , 

Ergo, the beginning of a project that would possibly 
serve as the catalyst that would liken the rapid advancement 
of educational technology to that of nuclear technology 
growth ensuing from the Manhattan Project of World War II. 

An Overview of the First 
Four Phases of TEC 

The TEC project was initially directed at resolving 
training deficiencies found in Army units for soldiers 
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serving in the Air Defense Artillery, Armor, Field Artil- 
lery, and Infantry branches. In order to isolate tasks that 
require training in the job environment (i.e. initial, 
refresher or continuation training) , a research project was 
initiated. The results of this study, conducted by the Army 
and by the Human Resources Research Organization (HumRRO) , 
served as the basis for identifying tasks for training in 
the early phases of the TEC project. 
TEC I 

Concurrent with the joint Army-HumRRO study, the U.S. 
Army Infantry School began a pilot test of the TEC concept. 
Due to the average readability level of soldiers, this test 
used a synchronized sound/slide format (35 mm slide and 
standard audio cassette) . Fifty-six lessons, applicable to 
Infantry noncommissioned officers, were developed by Infan- 
try School instructors and Infantry Officer Advance Course 
students. These materials were tested in ten Infantry batta- 
lions, representing both active army and reserve component 
units. 

The timing of this pilot test did not allow for a 
scientific derivation of tasks for which training would be 
required. Rather, subject matter was selected from 

4 
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existing documents which had not been validated against 
actual job performance requirements. 

Jacobs and Hardy (1974) investigated the effects of this 
pilot test of the TEC concept. Two significant findings 
which led to a continuation of the TEC project included; 

# The use of TEC I training materials proved effective 
in reducing the effect of verbal English ability on Military 
Occupational Specialty test performance. 

# Soldiers in units that required the use of TEC I 
training materials in study preparation for the Military 
Occupational Specialty test scored higher on the test than 
soldiers in units where TEC I training materials were not 
used* 

Other feedback on the TEC I project included a favorable 
attitude from unit leaders in regard to TEC as well as 
positive responses from TEC I lesson users. 

A significant finding impacting on the future of the TEC 
project was the storage requirements for the synchronized 
sound/slide lessons. Units simply did not have the capa- 
bility to stockpile and store volumes of lessons in their 
present format:, this finding led to new medium alternatives 
in the next phases of the TEC project. 

5 
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TEC II 

The results of the pilot test or TEC I had considerable 
influence on the next phase of the TEC project - TEC II. 
Many of the lessons learned from TEC I were instrumental in 
the design of the new methods TEC II employed in providing 
instruction for soldiers in Air Defense Artillery, Armor, 
Field Artillery and Infantry units. 

Among the new methods incorporated into the TEC II 
program were : 

0 Super 8mm film cartridges with synchronized audio 
cassette tapes replaced the 35mm slide/sound format. 

^ Lessons were based on the findings of a job/task 
analysis of Military Occupational Specialties for which 
instruction was to be developed. 

^ All lessons were subjected to at least one develop- 
mental trial and a validation test, using members of the 
intended TEC lesson users as subjects. 

0 A lesson development model was used to insure the 
application of educational technology to the TEC II projects 

The lesson development model adopted for TEC II is known 
as CISTRAIN, the acronym for Coordinated Instructional 
Systems Training. It was selected to advise government c\nd 
government contractor personnel on the educational techno- 
logy methods desired for application in the TEC program. 

6 
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Also, the model was to inrure a minimum quality of lessons 
early in the program and reduce the variability in the 
design of lessons that could have resulted from a less 
controlled approach. The CISTRAIN model is an empirical 
model of lesson development which relies heavily on tryout 
and revision cycles during lesson preparation and develop- 
mental testing as opposed to an analytical or artistic model 
which has been proposed by other instructional developers 
(cf. Bunderson and Faust, 1976). 

Lessons developed in TEC II were a joint effort between 
US Army service schools (the US Army Air Defense School, the 
US Army Armor School, the US Army Field Artillery School, 
and the US Army Infantry School) , two government contractors 
for lesson development (American Analysis, Inc., and Ins- 
group. Inc.), and one for audiovisual and audio lesson 
reproduction (Stockdale, Inc.). 

Success in TEC II was witnessed in many ways, to in- 
clude: verbal reports of supervisors noting improved 
overall professional and technical performance of Army 
service school personnel working on the TEC II project; 
positive reports from units where TEC II lessons were being 
put through development tests and validation trials; an 

7 
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improved communication between Army service schools which 
assisted in reducing redundancy in lesson preparation; 
increased reliability in Service School products expressed 
by unit using TEC II lessons; and a significant effect on 
the developmental testing and validation procedures for all 
instruction being produced in the schools participating in 
TEC II. 

These successes, along with an on-going needs assessment 
isolating additional support requirements for Army units, 
influenced the expansion of TEC while the TEC II phase was 
still gaining momentum, 
TEC III 

Following the introduction of TEC I and II, a survey 
administered in a representative sample of Army Combat Arms*^ 
unit commanders evidenced a need for training support in 
five high skill level jobs found in small numbers in every 
unit, A subsequent job analysis ascertained the specific 
and most important tasks for training assistance through 
what became known as the TEC III program. 



At that time there were four Army branches referred to 
as Combat Arms, They were: Air Defense Artillery, Armor, 
Field Artillery, and the Infantry, Late in 1975, Corps of 
Engineers was added as a Combat Arms Branch. 
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This expansion of TEC, TEC III, brought five^more Army 
service schools into the educational technology project 
which would be the catalyst for a significant change in how 
soldiers receive their training. 

While TEC I lessons had all been produced in a 35 mm 
slide/syncronized sound format, TEC II expanded media use to 
three new formats. They were, a super 8 mm film cartridge/ 
audio cassette, an audio cassette format, and the use of job 
aids. In TEC III, a fourth lesson format, printed text, was 
introduced. 

The media selection process in TEC I, II, and III was 
generally based on expected life of the lesson content and 
availability of tools and equipment for skill acquisition 
and practice. When possible, the audiovisual format was 
used in an effort to reduce the effects of low verbal ability 
on task mastery in other media as substantiated in earlier 
findings (Jacobs, T. 0., and Hardy, R. A. Jr., 1974). 
TEC IV 

Initial distribution of TEC II lessons was made during 
June 1974. The growing popularity and established useful- 
ness of TEC lessons caused commanders of combat service and 
combat service support units to request TEC lessons to 
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Schools added under TEC III included; the US Army 
Engineer School, the US Army Institute of Administration, 
the US Army Ordnance Center and School, the US Army Quarter- 
master School, and the US Army Signal School. 
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support training of their soldiers. Nine additional Army 
service schools were brought into the TEC program under the 
aegis of TEC IV. This increased the number of Army service 
schools in the TEC program to eighteen. 

During the initial stages of the TEC IV program, it was 
becoming evident that the Army service schools and contract 
monitoring agencies were internally benefiting from their 
efforts in the form of a sharply rising professional com~ 
petence in educational technology. The workshops, seminars, 
conferences, and daily job tasks required in this project 
were beginning to pay off in somewhat predictable forms. 

The reliance on a particular medium, in this case, 
audiovisual, was becoming less noticeable. Feedback from 
other media forms increased the confidence of the Army 
service schools to take a learning approach in TEC media 
selection, as well as that of lesson content, life expec^ 
tancy, and tool and equipment availability. 

During the TEC IV Program the Army began to be more 
discriminant in selecting tasks for training through the TEC 
Program (US Army TRADOC Letter, 1975). 
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New schools added under TEC IV included: the Academy of 
Health Sciences, the US Army Security Agency Training Center 
and School, the US Army Aviation School, the US Army Chaplain 
School, the US Army Intelligence Center and School, the US 
Army Judge Advocate General School, The US Army Missile and 
Munitions School, the US Army Military Police School, and 
the US Army Transportation School. 
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The contractual support for TEC IV progressed in two 
phases. with minor modifications, the first phase used the 
basic TEC contracting model (sequence of government and 
contractor actions) and procedures used in TEC II and III. 
During the first procurement of TEC IV lessons, a new lesson 
development contractor was added (Teaching Systems, Inc.), 
However, during the second procurement of TEC IV lessons, a 
significant change took place in the contracting module and 
procedures. Some of the major changes included: 

• One contractor (both development and reproduction) is 
used for the entire procurement rather than requesting a 
separate bid for each school. 

• The contractor makes an evaluation of government 
furnished materials (GFM) , and recommends and makes changes 
under a separately priced component rather than a research 
and analysis component. 

• Component pricing is in effect only until the govern- 
ment has approved a lesson's objectives, tests, content, 
design approach; and first draft script; fixed pricing then 
takes over^ 

• The requirement for use of the CISTRAIN Model was 
deleted. 

Justification for using one contractor for the second 
phase of TEC IV, which includes the projected development of 
over 300 lessons, was based on decisions regarding the 

11 
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management of the overall effort and the probability of 
reduced over-all costs in the program. Thereby, the govern- 
ment was applying cost-effectiveness techniques to the TEC 
program via the contractual apparatus. 

Throughout the' TEC program there has been repeated 
discrepancies in the quality of GFM, causing revisions of 
various quantity and quality to be made by both the govern- 
ment and the contractor. Because there was no standard rate 
or specific component in the contract addressing this type 
of evaluation or revision, it was decided to include an 
event in the contract calling for an evaluation and revision 
of GFM. 

Largely due to experience gained in TEC II and recommen- 

4 

dations made by analysts, the contract was modified from a 
total component price to a mixture of component and fixed 
pricing. This technique was implemented to reduce the over- 
all costs involved in the development of TEC lessons. 

Throughout the TEC program, up to and including the 
initial procurement for TEC IV, the CISTRAIN model had been 
specified in the contract statement of work as the method to 
be used in designing and developing TEC lessons. A decision 
was made in the second phase of TEC IV procurement to elimi- 



These analysts included personnel contracted through the 
Scientific Services Program of Battelle Memorial Laborator- 
ies, and government contract experts, both rrdlitary and 
civilian. -|^^ 
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nate this requirement. Use of CISTRAIN became optional, as 
did any other lesson development model proposed by the 
successful contractor and subsequently approved by the 
government. 

Influencing this decision was the increased sophistica- 
tion in the field of educational technology on the part of 
government personnel working in the TEC program. Possibly 
more important was the fact that component educational tech- 
nology contractors already have established lesson develop- 
ment procedures within their organizations. Their personnel 
are familiar with these procedures, use them in their func- 
tions of instructional design and development, and as long 
as their model met the criteria established for TEC lesson 
development, (i.e. performance objectives, criterion and 
performance tests, relevant materials only, developmental 
testing and revision, etc.), it was not necessary to pre- 
scribe or endorse a particular lesson development model. 

PROFESSIONAL SUPPORT USED IN TEC 
Beginning with TEC 's impetus, the Army has used a pro- 
fessional staff of analysts^ to provide continuous evaluation. 



These analysts have included: Allan J. Abedor, Lawrence 
T. Alexander, Theodore H. Blau, Frank W. Banghart, Walter R, 
Borg, Donald J. Cunningham, Stephen W. Daeschner, Robert H. 
Davis, William Deterline, Philip L. Doughty, Nancy L. Gulli- 
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state~of-the art updating, and to incorporate corrective 
measures when shortcomings have been identified. These 
personnel have been provided through a contract between the 
Army Research Office, Durham, North Carolina, and the 
Battelle Memorial Laboratories. This particular type of 
support has been invaluable to the growth and efficacy of 
the TEC program. 



Selected Data on TEC Lessons to Date 
As of April 4, 1976, there have been 471 TEC lessons 
validated. This means that they have been revised twice at 
a minimum and have either been fielded or are in the final 
stages of production. In Table 1 are scores from pre- and 
posttests given during large-group validations of these 
lessons . 
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TABLE 1. Validatior. Data on Trials Reported 
for TEC Lessons as of April 4, 1^76 
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(Cont'ci ford, Margo Hicks, Richard M. Jaeger, Roger A. 
Kaufman, Edward F. Kelley, , Raymond W. Kulhavy, Sidney S. 
Micek, Gail T. Rayner, Leonard Reiffel, Charles F. Schuller, 
David R. Stone, Ken B. Smith, Sanford Temkin, Donald T. 
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Methods for improving the techniques used in the devel- 
opmental and validation trials are constantly being sought. 
Evidence of this is found in the Signal School's adaptation 
of a sequential sampling technique (Epstein, 1975) which has 
a high confidence level and significantly reduces the number 
of subjects required in this phase of lesson development,^ 
Media Used in TEC 

Referred to earlier in this paper was the multimedia 
nature of TEC. In Table 2 is a synopsis of the media used 
in TEC lessons to date. 
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TABLE 2. Siurmary of Lesson Status and Media 
used in TEC Lessons as of March 29, 



(Cont'd) Tosti, Paul A, Twelker, Larry J, Walter, Walter 
A, Wittich and the Center for Educational Technology, 
Florida State University (through a separate contract) • 

^The normal procedure for the large group validation 
trials is to use an N of 30. 
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Numbers of Lessons in TEC 
Persons encountering literature on the TEC program for 
the first time are sometimes deceived into thinking it is 
just another "short range" project for a specific subject 
area. For those who may fit into this category, Table 3 
displays lesson status (completed & programmed) as of 
April 1, 1976. 



Number of 

Status of Lessons Lessons 

Lessons completed* 488 

Lessons in production 422 

Lessons currently being 

procured 319 

Lessons under consideration 

for future procurement (FY??) 630*** 

TOTAL 1,859 

*Lessons developed up to the developmental testing stage. 
**Projected for Phase II of the TEC IV procurement. . 
***Planning figures only. 



TABLE 3. Summary of the status and number 

of TEC Lessons as of April 1, 1976 



The recent acceleration in the number of TEC lessons 
ready for distribution or fielded is not discernible using 
Table 3. In order to emphasize the rapid growth 6f TEC 
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lessons available to soldiers in units another visualization 
is helpful. Table 4 depicts this expansion. 



250 




MAR MAY AUG OCT JAN APR 

75 75 75 75 76 76 



TABLE 4. Number of TEC Lessons 

ready for distribution or 
fielded at selected inter- 
vals 1975-76. 

Cost Data 

Very often there are questions concerning the cost of 
TEC lessons. Tempkin, et. al. (1975), conducted a cost 
assessment of Army training alternatives to include TEC. 
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Their data concerning the estimated cost of TEC lessons in 
the TEC II program only are shown in Table 5. 



Source 
at Combat Arms School 
at USACATB 

for Lesson Contractor 
for Lesson Master 
for Review 
Average Estimated 

TABLE 5. Estimated 
Lessons ( 



Cost 
$ 3,957.33 
493.31 
10,581.66 
814.91 
72.89 
$15,920.10 

al Cost of TEC II 
Tempkin, et. al. 1975) 



It is important to note that this estimation includes all 
identifiable costs. The cost for the lesson development 
only by a government contractor has ranged from a low of 

$4,800.00 to a high of $27, 000. 00 i Due to the partial 

> 

billing techniques used in this procurement, more definitive 
cost data are not presently available. Also, it should be 
pointed out here that each TEC lesson represents a 
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different size bite out of the Army's training needs, and 
therefore, costs will vary significantly from lesson to 
lesson when a component priced contract is used. Phase II 

of the TEC IV procureinent is attempting to somewhat stand- 

■ * J" 

ardize lesson costs by using a mixture of component and 
fixed pricing. 

TEC'S TRAINING EFFECTIVENESS 

As can be substantiated by the scores achieved during 
validation trials on posttests administered after one 
exposure to the TEC lesson, as evidenced in Table 1., TEC 
lessons adequately prepare the target audience for mastery 
of their objectives. Keeping these posttest scores in mind, 
a word of caution should be noted here. Caution should be 
taken in generalizing the instructional effect on pre to 
posttest achievement when these are identical measures. 
Tobias and Ingber (1976) found the correlation between these 
variables to be .84 in a similar study on achievement - 
treatment interaction. King (1973) stated earlier in TEC 
program evaluation that when knowledge test items are used 
for testing purposes, the relationship between a soldier's 
scores and actual field performance of the criterion task by 
the same soldier is unknown. Since many of the earlier TEC 
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lessons used knowledge tests, King's observation applies. 
Albeit, the intent of the program is to have performance 
tests for all TEC lessons, this desired characteristic has 
yet to be fully achieved. 

A study which measured actual criterion task performance 
of soldiers taking selected TEC lessons compared against 
soldiers receiving conventional instruction was completed by 
Knerr, Downey, and Kessler, (1975) . The results of the 
research indicated that TEC-trained groups in general scored 
highest on the performance tests. TEC lessons consistently 
improved soldiers' performance, regardless of their level of 
mental ability. This finding suggests that TEC lessons 
would be particularly useful for training of mixed-aptitude 
personnel. Further findings indicated, however, that where 
skill or practice with equipment is involved in the training 
for criterion task performance, conventional instruction is 
equally effective. 

Lessons under development in phase II of the;. TEC IV 
procurement will have performance tests administered as part 
of the contractual development process. This new addition 
to TEC lesson development adds a new dimension of evaluation 
one which promises data to substantiate and validate actual 
criterion task performance as a function of TEC lesson 
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instruction. In summary / TEC lessons are validating as an 
instructional technique for facilitating mastery of their 
objectives. There is evidence to confirm that they prepare 
soldiers to perform the actual criterion task even when 
knowledge posttesting techniques are used. Future TEC 
lesson development will take into account actual criterion 
task performance as a result of TEC instruction during the 
contractual development process. 

TEC EXPANSION IN FY 77 

Currently/ plans are being made to expand the TEC pro- 
gram to include one and possibly two new schools^in fiscal 
year 1977. This broadening of the program is projected to 
result in some 630 new TEC lessons for Army Training. These 
lessons will have a mixture of proponency in the new schools 
and with Army schools already in TEC. 



'The Defense Information School and possibly the Defense 
Language Institute . 
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FUTURE OF THE PROGRAM 



A look into the out-years foresees the TEC program to 
continue to grow as a' major training resource with the Army. 
With this growth there should be expected resistance which 
can be accounted for by the pendulum theory of social change. 
This theory asserts, in part, that ensuing a period of 
chcinge, there will be a period of reaction equal in magnitude 
to the magnitude of change (cf. Anderson and Faust, 1974, p. 
17). In the immediate future, TEC is envisioned as a program 
continuing in much the same media as it has to date. The 
number of lessons being produced should begin to level off 
during 1978. 

Looking farther into the 1980 's, the TEC plan is for a 
highly interactive wired-garrison training program which 
includes all types of training, to include both on- and off- 
line instruction (i.e. CAI, TEC, on-the-job training, corres- 
pondence courses). Computerized management of each soldier's 
training will allow for both immediate corrective feedback, 
on and advance placement into training modules. 

The hardware systems used in TEC during this period will 
be selected based on the research and development that takes 
place during the next few years. Software development will 
continue to use state-of-the-art educational technology 
applications. 
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EPILOGUE 



In an earlier paper on TEC (Roberts, Lawson, and Neal, 
1976), reference was made to the call for a Manhattan Project 
in Educational Technology made by Deterline (1970) to Gagne 
during an interview on the future of educational research. 
This call has been met by the Army through the TEC program. 
The use or systematic design and development approaches, 
employing developmental tryouts of instructional materials, 
has pro/en effective in the TEC program. Once again, the US 
Army has brought its expansive resources together in an 
effort to reduce a problem confronting the defense. This 
time the problem was not how to develop a weapon to end a 
world War. It was, how to best train onr soldiers to be pre- 
pared for the next war. As was the case with the Manhattan 
Project of the 1940's, there will be side effects, both short 
and long range, from the TEC project. These effects should 
be mostly positive, ones that allow for the improvement of 
human performance through the application of educational 
technology to instruction. 
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THE CONTRACTUAL LESSON 
DEVELOPMENT FLOWCHART 
FOR TEC - ADOPTED 
DURING PHASE II OF TEC IV 
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TRAINING EXTENSION COURSE (TEC) 
LESSON DEVELOPMENT FLOWCHART 
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EXPLANATION OF FLOW DIAGRAM 

EVENT 1 , DELIVERY OF GOVERNMENT FURNISHED MATERIALS (GFM) . 
The Government provides the Contractor with GFM for each 
lesson. At a minimum, the GFM consists of a Coordination and 
Contract Summary Sheet (CCSS) which includes (1) a subject 
area title, (2) purpose and scope, (3) target audience 
description, (4) references and equipment, (5) names of 
government technical advisor and/or subject matter experts, 
(6) minimum task performance standard, (7) training objec- 
tives, (8) criterion-referenced test items, and (9) necessary 
performance tests. Additional GFM such as previous lesson 
plans, field manuals, television tapes, technical manuals, 
photos, etc, is provided based on availability and perti- 
nence to the subject matter. The delivery of the GFM is 
accomplished at the appropriate US Army Service School. The 
Contractor reviews the GFM package at the school and re- 
solves apparent deficiencies with Government personnel. The 
Contractor and Government also make considerations and plans 
for the conduct of subsequent events. 

EVENT 2 . PREPARE LESSON CONTENT ANALYSIS REPORT. Upon 
receipt of the GFM, the Contractor prepares a Lesson Content 
Analysis. During this event the Contractor identifies 
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any deficiencies in the GFM which may interfere with devel- 
oping the Kit Design Approach in Event 4, determines the 
procedure to correct these GFM deficiencies and corrects the 
GFM deficiencies using these procedures. The Contractor 
uses this event to identify interrelationships and hierar- 
chical ordering of kits which are components of the lesson. 
Methods used in this event to perform the lesson content 
analysis may include; interviews, (with selected personnel 
from the target audience, subject matter experts, job incum- 
bents, supervisors, technical advisors, other personnel), 
observing performance of tasks at job or training sites, 
observing the administration of performance tests, observing 
on-going instruction of the training objectives and detail 
review of GFM. The methods used are in sufficient detail to 
gather the necessary information to prepare first draft 
materials in Event 6. The Contractor prepares a Lesson 
Content Analysis Report containing as a minimum: 

(1) GFM deficiencies noted by the Contractor and veri- 
fied by the Contracting Officer's Technical Representative 
(COTR) ; 

(2) Corrective action taken on GFM deficiencies; 

(3) When applicable, revised training objectives, cri- 
terion referenced test items, minimum standards, and perfor- 
mance tests; 
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(4) Identified number of kits with their inLerrela- 
tionships; 

(5) Identified media for each kit with its rationale; 

and 

(6) Modifications to the trial/validation procedures 
previously approved by the Government if required for the 
kits in question. 

EVENT 3 . GOVERNMENT REVIEW OF THE LESSON CONTENT ANALYSIS 
RF?ORT. 

The Lesson Content Analysis Report is submitted to the 
Government for review and approval. Approval authority 
rests with the COTR. If^ as a result of Government review ^ 
major defects are found to exists it may be necessary for a 
meeting beiL.ween (Contractor and Government personnel to 
correct r revise and/or change the material prior to the 
Contractor proceeding to Event 4 . 

EVENT 4 . PREPARE KIT DESIGN APROACH 

The Contractor prepares a Kit Design Approach for each kit. 
Each Kit Design Approach contains as a minimum: 

(1) Kit Description; A Kit Title ^ Technical Advisor/ 
Subject Matter Expert ^ Contractor's Kit Designer ^ and if 
more than one kit is being developed for a. lesson^ a state- 



ERIC 



40 

37 



ment describing how the kit being developed relates to other 
kits within the lesson, 

(2) Kit Strategy; A statement of how the kit is to be 
developed, it provides a^ overview of the complete kit to 
include what the student will be taught, prerequisites to 
the kit, sequencing or chaining and how each sequence or 
chain supports or accomplishes each training objective. It 
also provides the creative approach to the kit (music, 
hiomor, dramatic situation* motion, animation), 

(3) Kit Outline; A brief outline of the intended kit to 
include headings, tape running time, student involvement 
time, art/photo requirements and training objectives sup- 
ported by each outline heading, 

(4) Administrative Requirements; A brief outline of the 
requirements to support the kit (soldiers, equipment, 
materials, etc., and the specification requirements of the 
target audience, etc.), and 

(5) Change in Design; A provision for making and record- 
ing changes in the Kit Design Approach as it proceeds 
through the development process. The Contractor is required 
to utilize not only GFM, but also obtain information from 
Government Subject Matter Experts, Government Educational 
Specialists, and other appropriate Government personnel. 



41 

38 



EVENT 5 , GOVERNMENT REVIEW OF KIT DESIGN APPROACH 
The Contractor's Kit Design Approach is submitted to the 
Government for review and approval. Revisions and approval 
returned to the Contractor by the COTR via annotated Kit 
Design Approach and the Government identifies and provides 
each kit with a lesson number. Revision clarification as 
required by the Contractor is accomplished with the COTR. 
I£, as a result of Government review^ major defects are 
found to exist it may be necessary for a meeting between 
Contractor and Government personnel to correct ^ revise 
and/or change the material prior to the Contractor proceed- 
ing to Event 6. 



NOTE: If motion sequences are required and approved ^ the 
Government provides the appropriate motion sequence 
not later than Event 12 or specifies that the 
Contractor is required to shoot motion sequences 
for a particular kit. 



EVENT 6 . PREPARE FIRST DRAFT MATERIALS 

The Contractor prepares First Draft Script and thumbnail 
sketches with visual descriptors (Track 1) and/or prepare 
first draft manuscript and sketches with visual descriptors 
(Track 2) and/or prepare first draft script (Track 3). All 
drafts are prepared at a comprehension level suitable for 
the target audience. By comprehension levels it is meant 
that the target audience will be able to listen to^ 
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view, read, and interact with the materials in such a manner 
as to follow instructions, understand procedures, etc. as 
evidenced in the developmental trials and validation. 
(Events 11, 15 and 19) . 

EVENT 7 . GOVERNMENT REVIEW OF FIRST DRAFT MATERIALS 
The Contractor's First Draft Materials are submitted to the 
Government for review and approval. Revisions and approval 
are returned to the Contractor by the COTR via annotated 
First Draft Materials, if as a result of Government review, 
it is found that major problems exist in the First Draft 
Materials, it may' be necessary for a meeting between Con- 
tractor and Government personnel to correct, revise and/or 
change first draft content. 

EVENT 8 . PREPARE REVISED FIRST DRAFT AND VISUAL MATERIALS 
The Contractor insures all revisions, changes and/or correc- 
tions necessary as a result of Government review at Event 7 
are reflected in the Revised First Draft Materials. Semi- 
comprehensive storyboards (Track 1) Visual Dummy (Track 2) 
and the revised first draft script (Track 3) are prepared in 
this event by the Contractor. During preparation of these 
materials the Contractor prepares a pretest and a posttest 
to be used in the conduct of Events 11, 15 and 19. The 
pretest and posttest are submitted with the Revised First 
Draft Materials for review and approval. 
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EVENT 9 , GOVERNMENT REVIEW OF REVISED FIRST DRAFT AND 
VISUAL MATERIALS. 

The Contractor's Revised First Draft Materials^ visual 
materials, pretest and posttest are submitted to the Govern- 
ment for review and approval. Revisions and approval are 
returned to the Contractor by the COTR via annotated mater- 
ials submitted, or attached notes, references, etc. Revi- 
sion clarification as required by the Contractor is accom- 
plished by contact with the COTR. If as a result of Govern- 
ment review, it is found that major defects exist in Visual 
Materials submitted with Revised First Draft Materials it 
may be necessary for a meeting between Contractor and 
Government personnel to correct, revise and/or change sub- 
mitted materials. 

EVENT 10 . PREPARE MATERIALS FOR INDIVIDUAL TRIAL 
The Contractor insures that all revisions, changes and/or 
corrections necessary as a result of Government review Event 
9 are reflected in the materials required to conduct the 
Individual Trial. These materials may include, at the 
option of the Government, 35 mm slides of storyboards and a 
scratch tape of script (Track 1), revised visual dummy 
(Track 2) , and/or a scratch tape (Track 3) . At the 
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beginning of this event, the Contractor informs the COTR of 
the requirements for the conduct of Event 11. 

EVENT 11 A & B . CONDUCT INDIVIDUAL TRIAL 

The Contractor conducts the Individual Trial at a time and 
site agreed to by the Government. The Government monitors 
the conduct of the trial and verifies that approved proce- 
dures were followed. The location may be at the school 
responsible for the instruction or at another site when, for 
the purposes of obtaining valid test results, a more suit- 
able target population is available. This trial is con- 
ducted with 3 to 5 students. Testing instructional mater- 
ials on 3 to 5 students (individually) representing the 
target audience constitutes 1 trial. The Government pro- 
vides physical facilities for the conduct of the trial, to 
include 35mm projectors, tape cassette players and any 
military equipment for performance testing. The Contractor 
provides storyboards, 35mm slides in carousel trays, scripts 
and scratch tape (Track 1), visual dummy (Track 2), script 
and scratch tape (Track 3) as prepared in Event 10. The 
Contractor also provides any materials necessary for the 
proper certification of the trial, i.e., answer sheets, note 
pads, response arrays, etc., as required. The Contractor is 
responsible for its own administrative support during the 
trial. 
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EVENT 12 A & B , CONDUCT AND OBSERVE PERFORMANCE TEST 
The Government conducts the Performance Test which certifies 
that the student can in fact perform tasks learned from the 
instructional materials. This is necessary when the use of 
the actual equipment with the lesson is inappropriate 
because of size or the availability of equipment for train- 
ing purposes is extremely limited. The Contractor partici- 
pates as an observer during the conduct of the performance 
test for the purpose of identifying necessary kit revisions. 

EVENT 13 A&B . DETERMINE REVISION REQUIREMENTS 
Revision requirements are identified by the Contractor 
during the conduct of the trial in Event 11 and while observ- 
ing the performance test in Event 12. Revisions are speci- 
fied by the Contractor by annotation of the Revised First 
Draft Materials and by separate document keyed to the the 
script, storyboards^ visual dummy ^ etc. The Contractor's 
trial personnel remain at the trial site until an agreement 
is reached by the Government and the Contractor on the 
specific requirement for revisions^ changes and/or correc- 
tions to the instructional material following completion of 
the developmental trial and performance test. Determination 
is made by the Government at this event whether to use the 
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Small Group Trial (Event 15) or go directly to Group Valida- 
tion (Event 19) . The Government makes the decision at this 
event whether comprehensive storyboards or final art will be 
used for visuals. 

EVENT 14 , REVISE MATERIALS FOR SMALL GROUP TRIAL OR GROUP 
VALIDATION 

The Contractor incorporates the approved revision require- 
ments as determined in Events 13A and 13B and prepares the 
following: 

Track 1 - Revised Storyboards and Second Draft Script, 
(at the option of the Government) 35mm slides of storyboards 
and scratch tape; or 

Track 2 - Second Draft Visual Dummy, or 

Track 3 - Second Draft Script, and at the option of the 
Government a scratch tape. 

At the beginning of this event the Contractor informs the 
COTR of the requirements for the conduct of Event 15 or 
Event 19, 

EVENTS 15, 16, 17A, 17B, 18 , CONDUCT SMALL GROUP TRIAL 
The decision as to whether or not Events 15 through 18 are 
to be used is made by the Government during Events 13A and 
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B. The conduct of Events 15 through 18 are similar to 
Events 11 through 14 except that 6 to 8 students participate 
in the Small Group Trial. During the conduct of Events 15, 
16, and 17A and B, Government participation is the same as 
it was for Events 11, 12, and 13A, B. The Contractor has 
the same performance parameters and responsibilities in 
conducting the Small Group Trial as in the Individual Trial. 

EVENT 19 . CONDUCT GROUP VALIDATION 

The Group validations (all tracks) are conducted by the 
Contractor with 10 to 30 students. The actual number of 
students is determined by the Government. The Government 
monitors the conduct of the Group Validation and verifies 
that approved procedures were followed. The Government and 
Contractor's responsibilities for the conduct of this event 
are the same as those specified in Event 11. 

EVENT 20 . OBSERVE PERFORMANCE TEST 
For explanation see Event 12. 

EVENT 21A & B . DETERMINE REVISION REQUIREMENTS 

The Contractor, after completion of Events 19 and 20, 

finalizes all revision requirements. Revisions are speci- 
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fied by the Contractor by annotation of the Revised Draft 
Materials and by separate documents keyed to the script, 
storyboards, visual dummy etc. Since it is possible that 
instructional revision may be required, the Contractor's 
personnel remain at the trial site until an agreement can be 
reached by the Contractor and the Government on the specific 
requirements for revisions, changes and/or corrections to 
the instructional material following completion of the 
validation and performance testing. 

EVENT 22 . PREPARE FINAL DRAFT MATERIALS 

The Contractor, following Event 21A and B, incorporates all 
finalized, approved revision requirements into the instruc- 
tional materials for submission to the Government as a Final 
Draft Script and Storyboards (Track 1) or a Final Draft 
Visual Dummy (Track 2) or a Final Draft Script and Scratch 
Tape (Track 3) . During the accomplishment of this event the 
Contractor coordinates with the COTR the date and time for 
the conduct of the next event. 

EVENTS 23A & B . CONTRACTOR AND GOVERNMENT FINAL DRAFT 
REVIEW 

The Contractor and Government personnel attend a final 
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draft review with appropriate personnel present at the 
school responsible for the instruction. This meeting is to 
review materials and insure all final draft instructional 
materials are correct. This meeting results in directions 
explicit enough to preclude any misinterpretation or error 
from occuring in the production of the Master Materials 
(master tapes, final art, and camera-ready mechanicals). 
This review meeting represents a final date when all instruc- 
tional materials are reviewed and finalized. At this event 
the Government provides the information for the kit labels 
and Student Instruction Sheet (SIS) . 

EVENT 24 . PREPARE MASTER TRAINING KIT MATERIALS 

The Contractor using all approved Final Draft' Materials 

provides Master Training Kit Materials to include as. a 

minimum: 

Track 1 - Master art/photography, master narrated mag- 
netic tape (pulsed) , final script, shooting script; or 

Track 2 - Master camera-ready mechanicals (includes 
printed text and/or test sheets, answer sheets, etc., as 
required) , master art or retouched photos if not on mechan- 
icals; or 

Track 3 - Master narrated magnetic tape 
Final Script. 
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During the conduct of this event the Contractor and the COTR 
select a date for the review and approval which takes place 
at the Contractor's facility (Event 25) . 



EVENT 25 . INSPECTION OF MASTER TRAINING KITS 

At the Contractor's facility, the Project Engineer from the 

Naval Training and Equipment Center inspects Master Training 

Kits for quality, specification compliance, and compliance 

with approved Final Draft Materials. Following inspection, 

the Project Engineer accepts or rejects the Master Training 

Kits. 

EVENT 26 . PREPARE PREPRODUCTION SAMPLES 

The Contractor prepares (from the accepted Master Training 
Kits) a Super Smm color corrected film loaded in a Techni- 
" color Silent Super Smm Magi cartridge and a copy of the 
pulsed master tape in an appropriate tape cassette (Track 
1) , and/or proof prints of master mechanicals (Track 2) , 
and/or a duplicate of the master tape in an appropriate tape 
cassette (Track 3) . Following preparation of these mater- 
ials the Contractor prepares 4 copies of Track 1 and 3 along 
with the final approved script, and 3 copies of Track 2 to 
be delivered to appropriate facilities designated by the 
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Contracting Officer or a duly authorized representative. The 
Contractor has 8 working days to accomplish Event 26. 



EVENT 21 . GOVERNMENT INSPECTION OF PREPRODUCTION SAMPLES 
Upon receipt of the preproduction samples, prepared in the 
previous event, the appropriate school reviews them and 
sends their comments to the Project Engineer at the Naval 
Training Equipment Center. (NTEC) Orlando, Florida. The 
Project Engineer inspects the preproduction samples for 
compliance with applicable specifications and quality pro- 
visions of the contract. Upon completion of inspection the 
Project: Engineer notifies the Contractor through the COTR of 
acceptance or rejection and, if any, correctives action to be 
taken. This is followed by a document to the Contractor for 
record purposes. if correction is necessary, the Contractor 
takes corrective action as directed and resubmits the 
necessary items. (NTEC retains original samples for compar- 
ison to resubmitted items when received) . The Contractor has 
8 working days to resubmit the corrected samples. if no 
further corrective action or submissions are necessary and 
the materials are approved, the Project Engineer informs the 
Contractor through the COTR of this approval and initiates 
an Engineering Acceptance Report (EAR) which is forwarded to 
the Contractor to verify approval. For all tracks,, the EAR 
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sets forth a date of approval^ shipment required^ and 
destination. For Tracks 1 and 3 the EAR also specifies the 
quantities of kits required in mass reproduction. The 
Project Engineer labels the approved preproduction samples 
(Tracks 1 and/or Track 3) and returns them with the EAR. 
These samples are used by the Defense Contracts Administra- 
tion Services (DCAS) Inspector during the conduct of Event 
29. 

EVENT 28 . MASS REPRODUCTION 

The Contractor^ upon receipt of the approval in the previous 
event, packages and ships Track 2 Master Training Kits to 
the appropriate U. S. Army Service School. The Contractor 
includes printing instructions and annotated master mechan- 
icals to facilitate and insure that printing production can 
be without error. The packaging and shipping of these kits 
is in consonance with applicable specifications. The Con- 
tractor, upon receipt of the approval in the previous event, 
begins mass reproduction of Track 1 and 3 Master Training 
Kits. The Contractor has 20 working days, to reproduce and 
deliver the required number of copies. Approximately 10 
days prior to completion of the mass reproduction run the 
Contractor arranges with DCAS a date on which to conduct the 
inspection (Event 29) . 
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If a delay of 5 working days or more is forecast for the 
inspection, the Contractor informs the COTR of the delay. 



EVENT 29 . DCAS INSPECTION OF MASS REPRODUCTION ITEMS 
The Government performs an inspection of mass reproduction 
kits. Using the approved preproduction samples, applicable 
specifications, including packaging and shipping. The DCAS 
Inspector also insures mislabeling has not occured. If the 
Inspector finds that the kits do not comply with contract 
specifications there is corrective action taken by the 
Contractor to insure that all kits comply with the approved 
preproduction samples, specifications, etc. The Contractor 
initiates corrective action and supplies the reproduction 
copies within 20 working days. Requirements fot re-inspec- 
tion of corrected kits is coordinated with the DCAS office 
and inspection accomplished on all kits as outlined above. 

EVENT 30 . SHIPMENT OF MASS PRODUCTION ITEMS 
Following acceptance in Event 29, the Contractor packages 
and ships the mass reproduction kits to the designated 
facility, utilizing government (US Army) Indicia labels or 
Government Bills of Lading (GBL) as obtained from the local 
DCAS Transportation Office. The Mode of shipment is stipu- • 
lated by the Contracting Officer or a duly authorized 
representative . 
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